Practical Implementations of Principal Component Analysis

The Math behind PCA

Application 1 - Demographic Data

Dataset Description

This application was inspired by a paper published by UWF (Amin, Yacko, and Guttmann) on Alzheimer’s disease mortality (Tejada-Vera 2013) (Amin, Yacko, and Guttmann 2018).

The dataset is derived from US census data and contains demographic, health, and environmental metrics for counties in the United States. The dataset was filtered to select counties in the deep south.

Application 1 - Demographic Data

Check Assumptions: Linarity Analysis

The Harvey-Collier Test can automate this by running pairwise test for linearity, and if a significant nonzero slope on the residuals exists, the p-value will be less than alpha (Maureen, Oyinebifun, and Christopher 2022) (Harvey and Collier 1977).

Visually checking scatter plots is not a realistic method for inspecting linearity in real-world applications. Correlation plots are insufficient.

harvey_collier_test <- function(data, x, y) {
  formula <- as.formula(paste(y, "~", x))
  model <- lm(formula, data = data)
  test <- harvtest(model)
  p_value <- test$p.value
  return(signif(p_value, digits = 2))
}

variables <- names(data)
n <- length(variables)
p_matrix <- matrix(NA, n, n, dimnames = list(variables, variables))

for (i in 1:n) {
  for (j in 1:n) {
    if (i != j) {  # Avoid testing a variable against itself
      p_matrix[i, j] <- harvey_collier_test(data, variables[i], variables[j])
    }
  }
}

Application 1 - Demographic Data

Check Assumptions: Linarity Analysis (continued)

• The residual plot for a single pair that was flagged as nonlinear reveals why the Harvey-Collier Test declared nonlinearity.
  
• Transformations may risk altering other variable relationships, so no transformations were applied, acknowledging some information loss in PCA.

data_residual <- as.data.frame(data)

data_residual$residuals <- residuals(lm(Diabetes ~ obesity_age_adj, data = data_residual))

ggplot(data_residual, aes(x = obesity_age_adj, y = residuals)) +
  geom_point() + 
  geom_smooth(method = "loess", se = FALSE, color = "blue") +  # LOWESS curve
  geom_hline(yintercept = 0, linetype = "dashed", color = "red") +
  labs(title = "Residuals of Diabetes vs. obesity_age_adj",
       x = "obesity_age_adj", y = "Residuals") +
  theme_minimal()

Application 1 - Demographic Data

Check Assumptions: Outliers Analysis

note: the mean-centering and scaling is handled within the PCA implementation

• Outliers can distort PCA results by disproportionately increasing variance, shifting the direction of principal components, and inflating eigenvalues.
  
• Although PCA does not require normality, a roughly normal distribution minimizes the impact from outliers.
  
• Non-normal data may undergo transformations like the Box-Cox to approximate normality.
  
• Assessing skewness and kurtosis offers practical insights into distribution characteristics.

                   Kurtosis   Skewness
Atrazine_High_KG 866.377802 27.6792372
Mercury_TPY       72.724213  7.4402444
Lead_TPY          34.286296  5.0314443
Cancer             5.489565  0.3562768
Food_index         4.878495 -0.8643685
Heart_Disease      3.867908  0.8405355
Poverty_Percent    3.795263  0.4674652
obesity_age_adj    3.680773  0.2879323
Smoking_Rate       3.531763 -0.7543274
Diabetes           3.344971  0.5427094
SUNLIGHT           2.974585  0.3380277

Application 1 - Demographic Data

Check Assumptions: Outliers Analysis (continued)

• The Box-Cox transformation can normalize the data distribution, enhancing its suitability for PCA (“Compression of Spectral Data Using Box-Cox Transformation” 2014).
  
• The transformations create an approximate bell-shaped distribution, reducing outlier effects.
  
• The lambda values indicate minor power adjustments.

                 Lambda
Lead_TPY            0.1
Mercury_TPY        -0.1
Atrazine_High_KG    0.1

Application 1 - Demographic Data

Correlation Analysis

• While not a complete diagnosis, identifying highly correlated variables can indicate multicollinearity within the dataset (VIF will be looked at later).
  
• This provides insights to the effectiveness of PCA because correlated variables can be transformed into orthogonal components which eliminate multicollinearity.